Why

不知道為什麼，ILM沒有動作，沒將舊的indics刪除。
導致elastic直接報錯。

this action would add [2] shards, but this cluster currently has [1000]/[1000] maximum normal shards open;

緊急處理先將node的 shared開成1500

PUT /_cluster/settings
{
  "persistent": {
    "cluster.max_shards_per_node": 1500
  }
}

Solution

再來是檢查為什麼有那麼多shared沒刪除，
但一整批，一個一個刪除會瘋掉。
用wildcard卻會發生錯誤。

DELETE /qa-backend-2024.02.*

錯誤

{
  "error": {
    "root_cause": [
      {
        "type": "illegal_argument_exception",
        "reason": "Wildcard expressions or all indices are not allowed"
      }
    ],
    "type": "illegal_argument_exception",
    "reason": "Wildcard expressions or all indices are not allowed"
  },
  "status": 400
}

不想寫shellscript，一個一個撈index去刪除，
於是先開放wildcard吧。

PUT _cluster/settings
{
    "transient": {
        "action.destructive_requires_name": false // allow wildcards
    }
}

再來執行

DELETE /qa-backend-2024.02.*

結束。

ref. elasticsearch, how to delete multiple indexes with wildcard